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ABSTRACT 



This study was conducted to examine whether using a 
monolingual, simplified English dictionary as an accommodation on a reading 
test with limited-English-prof icient (LEP) Hmong students improved test 
performance. Hmong students were chosen because they are often not literate 
in their first language. For these students, bilingual dictionaries are not 
likely to be useful. Students for this study were drawn from, three urban 
middle schools in metropolitan Minneapolis. There were a total of 69 students 
in the non-LEP group, and 133 students in the Hmong LEP group. The study used 
a control group of non-LEP students and an experimental group of Hmong LEP 
students. All students were administered two reading passages with an English 
dictionary available, and two passages without a dictionary available, 
varying passage order and accommodation in both study groups . The students ' 
test performance on the two reading passages with dictionary accommodations 
was then compared to their test performance on the two passages without 
dictionary accommodation. The results showed that there was not a significant 
difference in reading comprehension scores for students in either the LEP or 
non-LEP group under accommodation conditions. However, it was found that the 
Hmong LEP students who reported using the accommodated (simplified) English 
dictionary showed a moderately significant gain. Issues discussed include 
student dictionary ability, dictionary interactions with test items, test 
development considerations, and current beliefs about dictionary 
accommodations and reading assessment. The pretest and posttest are included 
in appendices. (Contains 25 references.) (KFT) 
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Executive Summary 



This study was conducted to examine whether using a monolingual simplified English dictionary as an 
accommodation on a reading test with limited English proficient (LEP) Hmong students improved test 
performance. Hmong students were chosen because they are often not literate in their first language 
due to a lack of educational experiences in Hmong, which was first put into written form in the 1970s. 
For these students, bilingual dictionaries are unlikely to be useful. Thus, we studied the possible 
usefulness of a monolingual English dictionary for these students. Students for this study came from 
three urban middle schools in a large metropolitan area of Minnesota. There were a total of 69 
students in the non-LEP group, and 133 students in the Hmong LEP group. The study was conducted 
using a randomized counter-balanced design, with a control group of non-LEP students and an 
experimental group of Hmong LEP students. All students were administered two reading passages 
with an English dictionary available, and two passages without the dictionary, varying passage order 
and order of accommodation in both study groups. The students’ test performance on the two reading 
passages with dictionary accommodation was then compared to their test performance on the two 
reading passages without dictionary accommodation, using a repeated measures ANOVA procedure. 
Results showed that there was not a significant difference in reading comprehension scores for 
students in either the LEP or non-LEP group under accommodated conditions. However, it was found 
that intermediate level English proficiency students in the Hmong LEP group who reported using the 
P dictionary in the accommodated condition showed a moderately significant gain. Issues discussed 
include student dictionary ability, dictionary interactions with test items, test development 
00 considerations, and current beliefs about dictionary accommodations and reading assessment. 






^er|c 

<LV nup 



://education. umn.edu/nceo/OnlinePubs/LEPl. html 



2 



BEST COPY AVAILABLE 1 7/2001 



Effect of a Simplified English Language Dictionary on a Reading Test - NCEO LEP Report 1 Page 2 of 25 

Overview 



A challenge facing states and districts is determining best practices for including Limited English 
Proficient (LEP) students in their testing systems. One recommended approach is to provide testing 
accommodations that are designed to reduce language barriers while not changing what a test is 
supposed to measure. One accommodation that is available in several states is allowing LEP 
students to use a dictionary during a test. However, there are few studies on the effects of dictionary 
use on test performance and score validity. 

The way in which dictionaries are used as an accommodation on reading tests varies across the 
country. In a survey of state assessment directors for 1998-99, Rivera, Stansfield, Scialdone, and 
Sharkey (2000) identified 21 states that allowed the use of bilingual dictionary accommodations and 
only 3 states that specifically prohibited them. Among the states that allowed bilingual dictionaries, 1 1 
states allowed them on all assessment components and 1 0 states allowed them on some of the 
assessment components (Rivera, et al., 2000). According to a survey conducted by the Council of 
Chief State School Officers (Olson, Bond, & Andrews, 1999), which covered 1997-98 assessments, 
only two of these states allowed bilingual dictionaries and the third allowed an English language 
dictionary. 

Current views about the appropriateness of dictionary accommodations in testing differ among 
researchers, educators, and LEP students. For example, there are both research and opinions that 
support using dictionary accommodations. Researchers have shown that unfamiliar vocabulary may 
cause difficulty in understanding items on tests (Garcia, 1991) and that using dictionaries can help 
students’ reading comprehension (Goyette, 1996; Laufer & Hadar, 1997) and equalize skill 
assessments for LEP students (Rivera & Vincent, 1997). Further, LEP students and English as a 
Second Language (ESL) and Bilingual Education teachers have requested them in testing situations 
because dictionaries are an accommodation that students use in classes every day and will most 
likely use throughout their lives (Bensoussan, 1983; Liu, Spicuzza, Erickson, Thurlow & Ruhland, 
1997; Quest, Liu, & Thurlow, 1997). 

There is also research and opinion that opposes using dictionary accommodations. It is a common 
belief among some researchers and test developers that any alteration to the standard administration 
necessarily alters the validity of the test score. Others argue that dictionaries should not be allowed 
because they may negatively affect the validity of a test (Rivera & Stansfield, 1998; Spolsky, 1997). 
Other reasons given against using dictionaries are that students will need more time with dictionaries, 
that students may over rely on them (Roizen, 1984), and that they have not been found to 
significantly affect reading comprehension test scores of individuals learning English as a foreign 
language (Bensoussan, 1983; Nesi & Meara, 1991). 

One argument against dictionary use is that it negates the role of specific vocabulary knowledge as 
an essential component of reading ability (Bensoussan, 1983). However, Bensoussan argues that a 
student using a dictionary still needs to be able to successfully choose the right meaning of a word 
based on the context of a passage in order to correctly answer a test question. Contextual clues may 
not always be readily available in a passage to infer meanings of unknown words (in tests or in 
everyday reading). Therefore, the availability of a dictionary does not guarantee understanding of 
unknown words with or without sufficient contextual clues. 



According to some researchers, readers need to comprehend a certain percentage of a text to be 
able to infer meaning of unknown words. Laufer (1997) suggested 95% text comprehension 
(understanding of 3,000 word families) before reading skills in a reader’s first language will aid 
reading in the second language, including inferring meanings of words from context. Hirsh and Nation 
(1992) suggested 98% text comprehension for pleasure reading, requiring readers to have knowledge 
r r» Approximately 5,000 word families. Although reading language skills may be more developed in the 
fcKJL 
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second language of some students than in their first language, adequate text comprehension still 
requires understanding of a high percentage of the words. Also, the extent of word family knowledge 
needed in a reader’s “sight” or “automatic” vocabulary suggests that dictionary use during a reading 
comprehension test may not greatly enhance performance if students’ overall threshold vocabulary is 
too low. Observed problems with vocabulary thresholds have led some researchers to conclude that 
students’ problems with comprehension are basically lexical rather than due to lack of reading 
strategies (Haynes & Baker, 1993). However, a dictionary would not be able to compensate a student 
with great gaps in vocabulary knowledge, therefore its use may be more beneficial for students 
whose proficiency is near the level required for comprehending a text. 

In some studies conducted with students learning English as a Foreign Language (EFL) in Israel, it 
was found that dictionaries were more useful for students with a moderate level of English proficiency 
and some dictionary skills (Bensoussan, 1983; Laufer & Hadar, 1997), and that they did not benefit 
students with very low or very high proficiency. Therefore, the accommodation may not benefit the 
lower proficiency students who would need the most help (Shepard, Taylor, & Betebenner, 1998). 

The majority of dictionary accommodation reading studies have been conducted in EFL settings, thus 
it is important to study dictionary use in an English as a Second Language setting, especially where 
most students are not literate in their first spoken language. Further, because most dictionary studies 
have used EFL tests, our goal was to seek evidence to either support or refute the use of dictionary 
accommodations in large-scale tests in the United States. 

Researchers who support dictionary accommodations favor the use of bilingual or bilingualized 
dictionaries (English dictionaries with a native language translation) for both LEP students in the 
United States and LEP students overseas (Laufer & Hadar, 1997; Rivera & Stansfield, 1998). Laufer 
and Hadar (1997) found that, in general, monolingual dictionaries were the least useful 
accommodation on tests of English as a foreign language. However, in the United States where 
students are in an English as a second language setting, bilingual dictionaries may not always be the 
best choice, particularly for individual students who may not be literate or may not have received any 
education in their first language. Despite being classified as LEP, the language that these students 
read and write the most fluently may, in fact, be English. 

The quality and appropriateness of dictionaries (including the quality of their translations), whether 
bilingual or monolingual, vary greatly. Some bilingual dictionaries only give word for word translations 
or incomplete meanings, while others give definitions. English monolingual dictionaries, on the other 
hand, sometimes provide only the most basic definition of a word. If students are allowed to bring 
their own dictionaries to a test, some may be at a disadvantage because of the specific dictionary 
they choose. English language dictionaries tend to give more detailed explanation about the function 
of words in context; however, the number and depth of entries and illustrations are not equal across 
dictionary versions. There are also differences between regular and simplified English dictionaries. 
Abedi, Lord and Plummer (1997) studied the impact of simplified English in the test items themselves 
on the National Assessment of Educational Progress (NAEP), and found that simplified English on 
tests had a positive effect for all students, not just for those who are LEP. If simplified English has 
been found to affect students’ performance on tests, it is likely that simplified English in dictionaries 
could also have an effect. 



Goal of the Study 

This study was conducted to examine the possible effects of using a monolingual simplified English 

dictionary as an accommodation on a reading test with limited English proficient Hmong students. 

o 
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Research Questions 

Four primary research questions were posed for this study. 

1 . Do Hmong LEP students provided with a simplified English dictionary perform better than when 
the dictionary is not provided? 

2. How does use of a simplified English dictionary by Hmong LEP students affect the reliability of 
test scores? 

3. What are the characteristics of Hmong LEP students whose scores are most affected by the 
use of a simplified English dictionary accommodation? 

4. Do students (LEP and non-LEP) want to use a simplified English dictionary as a test 
accommodation? 



Method 

Participants 

Students for this study came from three urban middle schools in a large metropolitan area of 
Minnesota. There were a total of 69 regular education students in the non-LEP group and 133 
students in the Hmong LEP group. No restrictions were placed on the backgrounds of students in the 
non-LEP groups other than they not be LEP students or students receiving special education 
services. Some students in the non-LEP group were from Hmong language backgrounds, but none of 
them was receiving services for limited English proficiency. Students from both groups were recruited 
at all three schools; however, only two of the schools provided students from both groups. The third 
school chose only to allow participation for its Hmong LEP student population. Data on an economic 
status indicator (receiving free or reduced lunch) were collected for both the Hmong LEP group and 
the non-LEP group. The students in the two testing groups were comparable. 

Schools used similar ESL level designations (1-5), but the specific description of each level varied 
across sites. The levels indicate the range of students within ESL classes, from beginning to high 
levels of English proficiency. The number and percent of LEP students by level are presented in 
Table 1 (1=lowest proficiency level). 



Table 1. Number and Percent of LEP Students by ESL Level 



Group 


ESL Level 


Total 


1 


2 


3 


4 


5 


N 


5 


17 


36 


62 


10 


130 


% 


3.8% 


13.1% 


27.7% 


47.7% 


7.7% 


100% 
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Note: Level of English proficiency as assigned by school personnel. 



Design 

The study was conducted using a randomized counter-balanced design, with a control group of 
regular education students and an experimental group of Hmong LEP students. Students were 
administered two reading passages with the English dictionary available, and two passages without 
the dictionary. The students’ test performance on the two reading passages with dictionary 
accommodation was then compared to their test performance on the two reading passages without 
dictionary accommodation, using a repeated measures ANOVA procedure. 

The passages were designed to parallel Minnesota’s Basic Standards Reading Test, which is part of 
the state graduation exam. However, the items used for this study had not been used or equated with 
officially administered tests. The passages represent sample passages that had been reworked by an 
assessment specialist for use in the Minnesota Assessment Project study of bilingual reading test 
items (Anderson, Liu, Swierzbin, Thurlow, & Bielinski, 2000). In addition, an LEP graduation 
standards specialist at the Minnesota Department of Children, Families and Learning and a bilingual 
adult member of the ESL community provided advice on cultural background in the development of 
the test passages. The two halves of the test were divided into Form A and Form B. The passages 
were assigned to forms so that Form A and Form B had the same overall difficulty. Table 2 shows the 
study design, which is a modification of one design presented by Thurlow, McGrew, Tindal, 
Thompson, Ysseldyke, and Elliott (2000). 



Table 2. Study Design 





Hmonq LEP Students 


English-Speaking Regular Education ! 




Group 1 


Group 2 


Group 3 


Group 4 


Group 1 


Group 2 




Test 

1 


Form A 


Form B 


Form A 


Form B 


Form A 


Form B 






WithDictionary 


WithDictionary 


WithoutDictionary 


WithoutDictionary 


WithDictionary 


WithDictionary 


Wi 


Test 

2 


Form B 


Form A 


Form B 


Form A 


Form B 


Form A 






WithoutDictionary 


WithoutDictionary 


WithDictionary 


WithDictionary 


WithoutDictionary 


WithoutDictionary 


Wi 



Test Instrument 

Form A and Form B each contained two passages. Each passage was 900-1 ,040 words in length. 
For each reading passage, the examinee was asked 1 0 multiple-choice questions that addressed 
■£jr il i literal and inferential comprehension. The test used in this study was previously used in a study 
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on bilingual translation accommodations by researchers of the Minnesota Assessment Project. 
Evaluation of test score reliability indicated that it had the same or higher internal consistency as the 
actual Minnesota Basic Standards reading test for most of the test groups (Anderson et al., 2000). 

A dictionary exercise, developed by research staff with ESL teaching backgrounds, was also 
administered to determine each student’s proficiency with the simplified English dictionary. The 
exercise was composed of four questions. Two questions asked students to provide written 
definitions of words appropriate to context sentences that were provided to them. A third question 
was aimed specifically at alphabetizing skill. The fourth question was designed to determine students’ 
ability to use the dictionary for other information about parts of speech. A four-point scoring rubric was 
developed for raters to evaluate and score the exercise. 



Dictionary Accommodation 

A simplified English dictionary was used, as opposed to a Hmong Bilingual dictionary because most 
Hmong background LEP students have limited literacy in their first language. The dictionary chosen 
for the study was The American Heritage English as a Second Language Dictionary (Houghton Mifflin 
Company, 1998). This simplified English dictionary was chosen for several reasons. First, the 
dictionary for the study had to include more difficult academic words specific to the test passages 
(e.g., cum laude, precocious), and had to provide a range of definitions for words with multiple 
meanings (e.g., stand, produce). It also had to present the definitions at a simplified level of English 
for clarity, and yet fit the needs of a range of proficiency levels among LEP students. 

Some of the other available ESL dictionaries had more pictorial content, but had fewer of the words 
that had been identified as potentially problematic in the study passages. Other ESL dictionaries were 
too basic, providing only one meaning for words with multiple meanings, and often giving meanings 
that did not match the meaning of words in the test passages. Also, for one of the vocabulary items 
on the test, another dictionary provided a definition with an example sentence that could have misled 
students to choose an incorrect answer. The example sentence included extra descriptive information 
that was contradictory to the correct test response. 



Study Procedure and Timing 

Students were assembled into either an auditorium or classroom to take the test. The size of the 
groups ranged from 11 to 58 students per room. Students were first asked to fill out a brief pre-test 
questionnaire about language background to provide self-ratings of their English and Hmong 
proficiency in several modalities: speaking, listening, and reading. 

Next, students were administered one half of the test; some of the students getting the dictionary 
accommodation and some not. Students were allowed as much time as they needed to complete 
each half of the test, having been given a general time limit of two hours. After completing the first 
half of the test, the student raised his or her hand and a test administrator started the student on the 
second half. If the student had the dictionary for the first half, dictated by the color of the test cover, 
the administrator removed it; if the student took the first half without the dictionary, then it was 
provided to the student on the second half. During test administration, staff recorded start and finish 
times on the test covers at the end of each half of the test while providing and removing dictionaries 
for students’ use so that the time taken by students with and without dictionaries could be tracked, 
immediately after completion of the whole test, students were given a post-test questionnaire about 
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dictionary use during the test, their opinions on possible usefulness of an English dictionary on a 
reading test, and other background information on dictionary use and instruction in the classroom. A 
short dictionary exercise also was given after the post-test survey to determine levels of student 
ability in using a dictionary. 

Students were allowed approximately two-and-one-half hours to complete all materials. This time 
allotment was determined on the basis of schedule limitations in the schools. Following completion of 
the test, students either stayed in the same room or simply returned to class, depending on school 
requests. For example, at one site, the students who completed the test were given other activity 
sheets to work on while waiting for other students to finish before being released back to class. At 
other sites, students were allowed to leave after they completed the study materials. It is uncertain 
whether these varying procedures had any effect on test results. 



Results 

Proficiency in English and Hmong 

On student pre-test questionnaires, the control group and the Hmong LEP group answered a series 
of questions on language proficiency for speaking, listening, and reading in English and Hmong, and 
the length of time they had spent in U.S. schools. Most students in the Hmong LEP group reported 
higher reading ability in English than in Hmong (see Table 3), with most students rating their English 
reading ability as either “well” or “pretty well.” For speaking and understanding spoken English/ 
Hmong, the majority of the group described themselves between “Well” and “Pretty well” for English, 
and between “Very well” and “Pretty well” for Hmong. 



Table 3. Hmong LEP Group Self-Report for English and Hmong Reading and Speaking Ability 



Hmong LEP 
students 


Very Well 


Pretty Well 


Well 


Not Very 
Well 


Not well 
at all 


Total 


N 


% 


N 


% 


N 


% 


N 


% 


N 


% 


N 


% 


English 


Speak 


26 


20% 


64 


49% 


37 


28% 


5 


4% 


0 


0% 


132 


100% 


Read 


14 


11% 


67 


51% 


44 


33% 


7 


5% 


0 


0% 


132 


100% 


Hmong 


Speak 


69 


52% 


44 


33% 


15 


11% 


5 


4% 


0 


0% 


133 


100% 


Read 


10 


8% 


8 


6% 


25 


19% 


50 


38% 


40 


30% 


133 


100% 



In the non-LEP group (see Table 4), as might be expected, the majority reported their level of English 
proficiency as “Very well” in both modalities. This group did include some students with Hmong 
language background who were not limited English proficient. For these students, some reported a 
range of “Well” to “Very well” in understanding and speaking Hmong. However, very few reported 
reading Hmong above “Not very well.” 



Table 4 Non-LEP Group Self-Report for English and Hmong Reading and Speaking Ability 



Non-LEP 

students 


Very Well 


Pretty Well 


Well 


Not Very 
Well 


Not well 
at all 


Total 


N 


% 


N 


% 


N 


% 


N 


% 


N 


% 


N 


% 


O lqlish | Speak 


55 


80% 


10 


15% 


4 


6% 


0 


0% 


0 


0% 


69 


100% 


f r i 
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Read 


50 


73% 


14 


20% 


5 


7% 


0 


0% 


0 


0% 


69 


100% 


Hmong 


Speak 


10 


14.5% 


10 


14.5% 


2 


3% 


1 


1% 


46 


67% 


69 


100% 


Read 


0 


0% 


4 


6% 


5 


7% 


12 


17% 


48 


70% 


69 


100% 



Time in U.S. Schools 

Table 5 shows the length of time in U.S. schools for each group. The percentage of Hmong LEP 
students in U.S. schools 9 years or less (45.4%) was greater than the non-LEP group (1 1 .5%) (X 20 ) 
=24.9, pc.01). 



Table 5. Student Self-Report on Time in U.S. 





1-3 Years 


4-6 Years 


7-9 Years 


>9 Years i 


Total 


Count 


% 


Count 


% 


Count 


% 


Count 


% 


Count 


% 


Non-LEP 


0 


0% 


1 


1 .4% 


7 


10.1% 


61 


88.4% 


69 


100% 


LEP 


4 


3% 


25 


18.9% 


31 


23.5% 


72 


54.5% 


132 


100% I 



Reported Dictionary Proficiency 

Table 6 shows the result of students’ self report of monolingual English dictionary skills. All but eight 
students in the study (6.3%) described their dictionary skills as “Good” or above. However, a greater 
percent of non-LEP students described their skills as “Very Good,” while more LEP students 
described their skills as “Good.” (X 20 ) = 29.7; p<.01) 



Table 6. Self-Report on Using Dictionary for Non-LEP and LEP students 



How good are you at 
using an English 
only Dictionary? 


Very good 


Pretty good 


Good 


Not very good 


Total 


Count 


% 


Count 


% 


Count 


% 


Count 


% 


Count 


% 




Non-LEP 


44 


63.8% 


23 


33.3% 


2 


2.9% 


0 


0% 


69 


100% 




LEP 


38 


29.9% 


49 


38.6% 


32 


25.2% 


8 


6.3% 


127 


100% 



Dictionary Exercise Results 

For the dictionary exercise, raters independently rated each item using a four-point rubric, then 
scores were compared. The agreement between raters for scoring the exercise was 94%. 
Disagreements were resolved by rater consensus after discussion. 

The results of the dictionary exercise were somewhat problematic for identifying specific levels of 
skill. For example, in choosing a correct definition from words with multiple entries, there was an 
observed tendency for students to choose the first definition in the dictionary regardless of whether it 
fit the context of the example sentence. We believe that students’ scores do provide a good estimate 
of specific dictionary skills, such as alphabetizing, locating a word entry, and choosing the correct 
^nition. Given that the highest possible score on this exercise was 4.0, the mean scores for each 
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Table 7. Dictionary Exercise Mean Scores for Non-LEP and LEP Students 





Mean 


N 


Std. 

Deviation 


Minimum 


Maximum 


Non-LEP 


3.22 


67 


.69 


2 


4 


LEP 


2.72 


128 


.83 


0 


4 


Total 


2.89 


195 


.82 


0 


4 



Note: One point was given for each correct answer (total possible 4). Half points were allowed on some items. 

We also compared dictionary exercise results to students’ self-ratings of dictionary ability. Table 8 
shows the average score on the dictionary exercise with each self-reported dictionary proficiency 
rating. Generally, the Hmong LEP students’ performance increased with their self-ratings of dictionary 
proficiency, but the non-LEP students’ performance was nearly the same across each self-rated 
proficiency level. 



Table 8. Students’ Self-Report on Dictionary Ability by Dictionary Exercise Means 



Dictionary Exercise 


Non-LEP Self Rated Ability in Using Dictionary 


Very good 


Pretty 

good 


Good 


Not very good 


Not good at all 


Total 


Mean 


3.21 


3.24 


3.25 


None 


None 


3.22 


N 


42 


23 


2 


None 


None 


67 


Std. Deviation 


.75 


.62 


.35 


None 


None 


.69 




Hmonq LEP Self Rated Ability in Using Dictionary 


Very good 


Pretty 

qood 


Good 


Not very good 


Not good at all 


Total 


Mean 


3.06 


2.74 


2.3 


2.81 


None 


2.72 


N 


35 


47 


32 


8 


None 


122 


Std. Deviation 


.73 


.82 


.82 


.75 


None 


.83 



Test Results for Accommodated vs. Non-Accommodated Performance 

Initial analyses were conducted to determine whether there were possible effects for form or the order 
the accommodation was given. This involved looking at the mean performance on each half of the 
test (20 points possible on each half of the test) and the mean for the entire test for LEP and non-LEP 
groups by the order of form (i.e., Form A and Form B) and order of administration (accommodation 
given on first half vs. accommodation given on second half). Table 9 shows that there was no effect 
for the Form that students received with the accommodation (Fo.iss) =.12; p =.73). 



Table 9. Effect of Accommodation by Form 





Mean Score 


Std. 

Deviation 


q '"roup 

ir 


Non-LEP 


Dictionary on Form A 


25.9 


7.09 
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Dictionary on Form B 


26.5 


7.98 




LEP 


Dictionary on Form A 


18.8 


6.36 






Dictionary on Form B 


18.9 


7.15 



To examine the possibility that the order in which the accommodation was administered made a 
difference on test performance, we compared test scores for those receiving the accommodation on 
the first half with the scores of those receiving the accommodation on the second half. The mean 
score for the LEP group with the accommodation on the second half of the test was 1 9.4, whereas 
the mean score with the accommodation on the first half was 18.3. A 2x2 ANOVA was run with Group 
and Order treated as fixed effects (see Table 10). The effect for the order of accommodation was not 
significant (F a,i98)=.043; p=.73). 



Table 10. Effect of Accommodation by Order 





Mean Score 


Standard 

Deviation 


Group 


Non-LEP 


Dictionary on 1 st half 


26.5 


8.07 


Dictionary on 2 nd half 


25.9 


6.92 


LEP 


Dictionary on 1 st half 


18.3 


6.65 


Dictionary on 2 nd half 


19.4 


6.87 



The means shown in Table 1 1 indicate that regardless of the order the accommodation was given, 
students performed better on the first half of the test than on the second half (Fo. 2 oo)= 22.1;p<.01). 



Table 1 1 . Student Performance on Both Halves of Test 





1 st Half mean 


2 nd Half mean 


Non-LEP 


13.8 


12.4 


LEP 


9.9 


9.0 



The main research question in the study was whether the use of a simplified English dictionary would 
improve test performance, and whether improvement would be greater for the LEP group than for the 
non-LEP group. A repeated measures ANOVA was run wherein the within group variable had two 
levels: (1) test score without accommodation, and (2) test score with accommodation. The between 
group variable was LEP status (LEP vs. non-LEP). The difference between these two groups was 
significant (F i,2oo=49.7; pc.01). The effect for the accommodation was not significant (F i,2oo=.15; 
p=.70), and the interaction also was not significant (F i,2oo=.39; p=.54). Table 12 shows the cell means 
and 95% confidence interval for each condition by group. The non-LEP group performed the same 
under both conditions whereas the LEP group performed slightly better with the dictionary 
accommodation. 



Table 12. Student Performance With and Without Dictionary Accommodation 
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Group 


Non-LEP 


Dictionary 


13.1 


.48 


12.13 


14.02 


No Dictionary 


13.1 


.46 


12.23 


14.03 


LEP 


Dictionary 


9.6 


.34 


8.88 


10.24 


No Dictionary 


9.3 


.33 


8.66 


9.96 



Because not every student in either the control or Hmong LEP group self-reported using the 
dictionary accommodation when it was provided, a second analysis was conducted for only those 
students who reported that they had used the accommodation. In this analysis we compared the 
performance with and without the accommodation by self-reported English language proficiency. The 
analysis indicated that LEP students who self-reported lower English proficiency (“well” to “not well at 
all”) did not benefit as much from using the dictionary accommodation as LEP students who self- 
reported higher English proficiency (“pretty well” to “very well”). This high English proficiency group 
scored an average 1 .2 points higher using the accommodation compared to performance without the 
accommodation; the lower proficiency group performed nearly the same under both conditions (see 
Table 13). This interaction between proficiency level and condition was significant ( Fo,97)=4.78; 
p=.03). 



Table 13. Student Performance by Accommodation Conditions and Self-Reported English Reading Ability 





Mean 


Standard 

Error 


95% Confidence Level 


Lower 

Bound 


Upper 

Bound 


English Reading 


Very to 


Dictionary 


10.759 


.481 


9.805 


11.713 


Ability 


Pretty well 


No Dictionary 


9.569 


.483 


8.610 


10.528 


(dichotomized pre-test 


Well to 


Dictionary 


8.220 


.572 


7.085 


9.354 


Question 6) 


Not well* 


No Dictionary 


8.488 


.574 


7.348 


9.628 



A similar analysis was conducted for the non-LEP group using self-reports of dictionary use during 
the test. It showed that reading score means in this non-LEP group were not affected by the 
dictionary accommodation (Fo.e?) = .61 ; p=.44). 



Affect of Accommodation on Test Reliability 

Test score reliability was estimated using Chronbach’s alpha, a measure of internal consistency. This 
statistic can range from 0.0 (completely unreliable) to 1 .0 (perfectly reliable). Achievement tests are 
designed to obtain a reliability of about .85. We computed the reliability for each half of the test (20 
questions each) taken under both accommodated and non-accommodated conditions. Because 
reliability is affected by test length, we anticipated that the reliability for each half of the test would be 
slightly lower than .85. Table 14 below shows Chronbach’s reliability coefficient for each half of the 
test by condition, accommodated and non-accommodated. 

Table 14 Chronbach’s Alpha for Test Reliability 



i 


Test Items 


r 


i i 1 



miamam 
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Condition 


First half 


Second half 


Dictionary 


.79 


.76 


No dictionary 


.73 


.79 



Performance on Test’s Vocabulary Items 

A closer analysis was conducted on the test results of specific vocabulary items to determine whether 
performance gains with the dictionary accommodation could be attributed to performance on two 
questions that required only the definition of a term. There was one vocabulary item on each half of 
the test so that one half of the students would have encountered each word under accommodated 
conditions. The item questions were as follows: 

Items 8 and 28 The term precocious used in the title means: 

Items 5 and 25 As used in this article, the word produce is best defined as: 

The dictionary entries available to students (if in the dictionary accommodation condition) are 
reproduced below (without pronunciation information): 

Precocious, adj. Showing mental skills or abilities at an earlier age than is normal: He was a 
precocious child who learned to read at three. . . . 

Produce. Tr.v. 1 . To bring forth (sthg.): Seeds grow up to produce plants. 2. To create (sthg.) by 
mental or physical effort: produce a painting. 3. To manufacture (sthg.): produce parts for 
machines. 4. To cause (sthg.) to exist: Industrial growth produced a new kind of business 
organization. 5. To supervise and finance the public presentation of (a movie, for example): 
produce a play.- n. (U) Farm products, especially fruits or vegetables: Excuse me, where can I 
find the produce? 

It should be noted that “precocious” was not embedded in any of the text of the passage but was 
presented somewhat independent of the context in the title, “Precocious preteen is youngest college 
graduate.” Students would need to have understood its part of speech and rhetorical function in the 
title and connected that to the overall description of the boy’s qualities in the passage to infer the 
meaning of the word. The “produce” item, in contrast, occurred with relatively high frequency in the 
passage, embedded in sentences with numerous descriptors of fruit and vegetables. 

To investigate whether the performance on these vocabulary items was enhanced by dictionary use, 
thus potentially responsible for part of the average gain, we calculated chi-square results of number 
correct by school-assigned ESL level and accommodation condition with these vocabulary items (see 
Table 15). Only LEP students who reported using the dictionary for at least a few words were 
included. 



Table 15. Percentage Passing ‘Precocious’ Item. 



Precocious Item Percent Correct 


ESL Level 




1,2,&3 


4 & 5 




43.5% 


61 .54% 


Dictionary 


(N=10) 


(N= 16) 


ir 


52.2% 


33.3% 
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No Dictionary 



(N= 12) 



(N= 8) 



Statistics for table: (X 2 ^)=2.10; p=.15) 

Table 15 shows that for the item asking students to select the correct definition for “precocious,” lower 
ESL level students did slightly less well on this item with the accommodation available than did those 
without it. For higher ESL level students, 50% more students got the correct answer with the 
accommodation (N=16) than without it (N=8). It is uncertain why the lower level group without the 
accommodation was able to do better than the higher level group (52.2% vs. 43.5%). Even when the 
correct single meaning dictionary entry was available, some students did not choose the correct 
answer, regardless of school assigned ESL level. 

Table 16 shows that lower ESL level students did slightly better with the dictionary accommodation on 
the “produce” item than without the accommodation. The higher level students did slightly better 
without the accommodation than with the accommodation. 



Table 16. Percent Passing ‘Produce’ Item 



Produce Item Percent 
Correct 


ESL Level 


1,2,& 3 


4 & 5 


Dictionary 


39.3% 


41.2% 




(N= 11) 


(N= 10) 


No Dictionary 


36.4% 


57.7% 




(N= 8) 


(N= 15) 



Statistics for table: (X 2 ^=1.38; p=0.24) 

Reported Dictionary Use 

Table 17 shows results of post-test self reports from students about their overall dictionary use during 
the test. Students in the Hmong LEP group reported using the dictionary more than the non-LEP 
group. For new words, the Hmong LEP group reported more use (73%) than the non-LEP group 
(46%). However, the percentage of students who used the dictionary to check words they already 
knew was similar for the Hmong LEP group (12%) and the non-LEP group (7%). The third question 
about dictionary use was included in case students used the dictionary in a way that was different 
from the ways we thought it would be used during the test (e.g., looking up part of speech 
information, pronunciation, etc.). We are not able to say how the students defined “other information” 
in this question, but this may be a useful area for further study. 



Table 17. Student Self-Report on Dictionary Use During Test 
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Questions 


Group 


Yes 


No 


Total 


N 


% 


N 


% 


N 


% 


Did you look up new words? 


Non-LEP 


32 


46% 


37 


54% 


69 


100% 


LEP 


93 


73% 


34 


27% 


127 


100% 


Did you look up words you 
already knew? 


Non-LEP 


5 


7% 


64 


93% 


69 


100% 


LEP 


15 


12% 


112 


88% 


127 


100% 


Did you look up other 

information using the 
dictionary? 


Non-LEP 


15 


22% 


54 


78% 


69 


100% 


LEP 


68 


53.5% 


59 


46.5% 


127 


. 100% 



We analyzed self-reports according to which passage(s) students said they used the dictionary. 
Although the Hmong LEP group reported a larger overall percentage of dictionary use for each 
passage than the non-LEP group, the two groups showed the same pattern of use for the four 
passages (see Table 18). 



Table 18. Student Self-Report of Using Dictionary by Reading Passages in Test 





Hmong LEP 


Non-LEP 


N 


% 


N 


% 


Passage 1 


30 


24% 


4 


6% 


Passage 2 


59 


47% 


22 


32% 


Passage 3 


40 


34% 


7 


10% 


Passage 4 


42 


36% 


7 


10% 



Most students reported using the dictionary for Passage 2, which contained the question on defining 
“precocious.” Passage 3 contained the item on defining “produce.” It may be that Passage 2 had 
more reported dictionary users because the word “precocious” is probably more difficult for students 
in both groups than is the word “produce” (fruits and vegetables) found in Passage 3. Both groups 
also reported using the dictionary less for Passage 1 and about equally for Passages 3 and 4. 



Use of Dictionary Accommodation During Test as a Function of Reported English 
Proficiency 

Table 19 shows whether LEP students who reported lower English proficiency used the dictionary 
more or less than students reporting higher English proficiency. Findings showed, as may be 
expected, that more students who reported higher language proficiency also reported that they did 
not use the dictionary accommodation. Still, 14.6% of the students in the lower proficiency group 
reported not using the dictionary at all. 



’’-‘-'a 19. Reported English Ability by Reported Dictionary Use During Test 
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Two self reported proficiency levels for 
reading in English 


Self reported dictionary use 


Total 


Did not use 


Used for few words 


Used for some to all 
words 


How well do 
you read 
English? 


Very well to 
Pretty well 


Count 


20 


30 


28 


78 


% 


25.6% 


38.5% 


35.9% 


100% 


Well to 
Not well* 


Count 


7 


18 


23 


48 


% 


14.6% 


37.5% 


47.9% 


100% 


Total 


Count 


27 


48 


51 


126 


% 


21% 


38% 


41% 


100% 



*No student self-rated “not well at all.” 

Which Characteristics Predicted Overall Test Performance? 

A multiple regression analysis was conducted to assess which factors may account for differences in 
the performance of LEP students. Factors included in this analysis were gender, English reading 
ability self report, years in U.S., ESL level, difference in time on the test half with accommodation and 
the test half without accommodation, and dictionary exercise scores. After accounting for these 
variables, only school-assigned ESL level and dictionary exercise score accounted for a significant 
proportion of the variance in test scores. 

We also wanted to determine which characteristics, if any, were associated with the test score gains 
on accommodated items. A multiple regression analysis was run with the following predictors: gender, 
reported English reading ability, time in U.S., ESL level, time differences on the test half with 
accommodation and the test half without accommodation, and dictionary exercise scores. For the 
LEP students who showed the most gain on accommodated items, the only significant correlation 
was for time difference between the halves of the test with and without accommodation. LEP students 
who spent more time on the half of the test with the dictionary accommodation than on the half 
without the dictionary accommodation performed better under the accommodated condition (r=.257, 
p=.006). To test whether these students might have “rushed” the half without the dictionary, we 
compared the mean number of minutes spent on the half without the dictionary for the LEP students 
showing a gain compared to those who did not show a gain. For greater specificity, we looked at 
three groups: (1) students who lost at least 3 points on the accommodated half, (2) students whose 
difference between halves was 2 points or less, and (3) students who gained at least 3 points on the 
accommodated half (see Table 20). We found no significant difference in the amount of time spent by 
each group (F 2,124=1.28; p=.28). 



Table 20. Test Time Means for Three Groups (Non-Accommodation Condition) in Minutes 





Lost at least 
3 points 


Difference 
of 2 points 


Gained at 
least 3 points 


Total 


Mean 


33.5 min 


35.9 min 


32.9 min 


2092.9 


N 


22 


79 


26 


127 


Std. Deviation 


484.5 


612.2 


495.9 


571.44 



We also found that the correlation between testing time and performance, regardless of whether the 
student was in the dictionary accommodation condition, was not significant. These results suggest 
that students taking full advantage of the accommodation showed statistically significant performance 
npins even after controlling for other variables. 
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Even though the dictionary exercise scores did not correlate significantly with LEP student 
performance, the gain that correlated with more time spent during the accommodation condition may 
actually have been due to increased time with dictionary use. 



Analysis of Testing Time 

We found that both groups (LEP and non-LEP) spent more time on the first half of the test overall, 
regardless of the order in which they received the accommodation. Also, the non-LEP group spent an 
average of 4 minutes more time on the first half of the test than did the LEP group, whereas the LEP 
group spent an average of 6 minutes more time on the second half than did the non-LEP group. 



Opinion on Dictionary Usefulness 

Even though 34.5% of students reporting high English proficiency did not report using the dictionary 
in the two test groups, most students in the non-LEP group (84.1%) and the Hmong LEP group 
(95.8%) reported that access to a dictionary would be helpful on a reading test. 

Dictionary Type, Usage, and Instruction 

The two groups also responded similarly to questions about instruction in dictionary use and 
classroom usage (see Table 21). A majority in both groups had been instructed in the last year on 
how to use a dictionary (non-LEP, 66.7%; Hmong LEP, 65.3%), and had used them in classes (non- 
LEP, 79.7%; Hmong LEP, 87.3%). The type of dictionary used in classes for both groups was a 
monolingual English dictionary (non-LEP, 92.8%; Hmong LEP, 90.7%). The fact that there was a 
lower reported ability level in reading Hmong language than in reading in English helps explain why 
bilingual dictionaries are not used as often (4.2%) in classes for these students. 



Table 21. Type of Dictionary Used in Classes 



What kind of dictionary 
do you use in classes? 


English 


Bilingual 


Other 


None 


Total 


N 


% 


N 


% 


N 


% 


N 


% 


N 


% 


Non-LEP Group 


64 


93% 


2 


3% 


2 


3% 


1 


1% 


69 


100% 


Hmonq LEP Group 


107 


91% 


5 


4% 


4 


3% 


2 


2% 


118 


100% 



Discussion 



The primary goal of this study was to determine whether providing a monolingual English dictionary 
would improve the test performance of LEP students. A repeated measures ANOVA indicated that, 
overall, the effect of the monolingual English dictionary accommodation was not significant. In other 
words, test performance for LEP students was about the same under both standard and 
accommodated conditions. Not all students reported using the accommodation when it was provided. 
For those students who reported using the accommodation, LEP students with intermediate self- 
O jrted English reading proficiency had a statistically significant test score gain due to the presence 
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of the dictionary accommodation. Hmong students with poor English-reading proficiency did not 
appear to benefit from the accommodation even when they made use of it. This result is consistent 
with findings that lower and higher level proficiency students in EFL settings do not benefit from 
dictionary accommodations on reading tests as much as intermediate level students (Bensoussan, 
1983; Laufer & Hadar 1997). It is interesting to note that even though the accommodation did not 
result in a significant overall gain for LEP students, 95.8% of the LEP group believed that providing an 
English dictionary would be helpful on a reading test. It is unknown whether this discrepancy is 
related to the fact that students knew the test used in the study did not actually count. They may have 
believed that dictionaries would be helpful on a test that actually counted. 

Anytime a test is administered under non-standard conditions, there is concern that test score validity 
may be compromised. We assessed this possibility by examining the correlation between item 
difficulty for LEP students under standard and accommodated conditions, and item difficulty estimates 
for the non-LEP group without the accommodation. The correlations were similar. In this study, it 
would appear that the presence of the accommodation neither enhanced nor compromised the 
validity of the test scores. 

The question of how the added lexical resource of a dictionary may have interacted with test items 
should also be addressed. For example, the results of the two vocabulary items on the test appeared 
similar to the dictionary exercise results where an observed tendency was for students to choose the 
first definition in the dictionary to answer a question. It is possible that if students with this tendency 
used the dictionary for these items “precocious” would have more correct answers because it had 
only one entry and the correct answer was therefore listed first, whereas “produce” had several 
definitions with the correct one listed last. However, there may be more complex reasons for the 
difference in performance for these two items such as the availability of contextual cues, ability to 
infer meaning, dictionary skill, item distractors, or too low a vocabulary threshold for efficient 
inferencing or dictionary use. 

Although we do not know whether students used the dictionary for these specific items, the results do 
show that the availability of a dictionary did not guarantee that students would be able to answer 
these vocabulary items correctly. Further, we do not know whether the dictionary accommodation 
was possibly disadvantageous for students on at least one of the items. 

While the results of this study are consistent with Bensoussan’s point that using a dictionary on a 
reading comprehension test does not give away answers (even on some vocabulary items), the 
validity of items still depends on how the item choices interact with dictionary entries and the context 
of the test passage. For vocabulary items, if distractors include options that are not other meanings or 
plausible meanings of words, or if one of the options closely matches a dictionary’s entry (as was the 
case with “precocious” in our study), an item is potentially testing only basic dictionary skill and visual 
word matching. In contrast, if vocabulary item choices included multiple meanings of a word (i.e., our 
“produce” item), then the student would need to use more sophisticated reading and dictionary skills 
to determine the correct answer. It is important that test items are written to appropriately narrow the 
reading abilities being tested, and that where a dictionary accommodation is being considered, this be 
taken into account in the development of the reading test items. For reading comprehension items, it 
may be similarly argued that dictionaries do not “give away” answers, because students still need to 
make use of vocabulary knowledge using their reading strategies or strategic competence. For 
example, looking up the word “fact” or “opinion” in the wording of a comprehension question is not 
going to give students the correct answer requiring them to identify an opinion from a passage (i.e., 
Which of the following is an opinion, not fact, expressed in the article?) 

The results of this study are, to some extent, confined to tests similar to the one used in our study. It 
was a basic skills test that used newspaper-like passages and asked questions about them. Context 
O' ' important in providing cues for students to answer questions. Dependence on context reflects an 
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overall trend in performance-based testing, as well as in second language reading and vocabulary 
testing. When a test is designed to test specific vocabulary where context is not needed to choose the 
correct definition, then access to a dictionary could very well invalidate students’ test scores. Despite 
criticism that discrete point vocabulary items stress knowledge of words without having to relate them 
to the context in which they are used (Read, 2000), these types of items do still appear in tests. When 
that is the case, allowing the use of a dictionary accommodation probably is not appropriate. 

An accommodation option that is similar to dictionaries is word glossaries, which may provide a way 
to control for the varied definitions across different types and skill levels of dictionaries. Choosing 
items for glossaries may prove difficult for students with certain backgrounds. For example, Hmong 
students may need some Latin based words glossed that a Spanish language background student 
may not, and some Latin based words in English may be false cognates for the Spanish student. 

Also, there may be conceptual differences between language groups in general background 
knowledge and experiences that would need to be considered in developing glossaries. Earlier 
research (Brutten, 1981) showed that experienced teachers could predict to some extent words that 
would be problematic for students in a reading text. This suggests that glossaries might need to be 
individualized. There would still need to be consensus on how glossaries or dictionaries fit into the 
definition and demonstration of vocabulary knowledge and reading abilities for assessment purposes. 

An issue raised by some researchers is that students may over-rely on dictionaries in testing 
situations. In this study, we did not collect information on how students used the dictionary on specific 
items or their attitudes toward taking the test with the dictionary accommodation related to test 
preparation, so it is unclear whether some students may have over-relied on the dictionary. Generally, 
over-reliance on the dictionary during the test was not observed by staff administering the test or in 
the students’ own self-reports, which showed that most LEP students used the dictionary for 
“few” (37.8%) or “some” of the words (32.3%). Only two (1 .6%) of the LEP students reported using 
the dictionary for all the words, and 22% of the LEP students reported not using the dictionary at all. 
Of course, students did realize that the test used in the study did not actually count; this may have 
affected the extent to which they used the dictionaries. 

Although most studies on dictionary accommodations have recommended bilingual or bilingualized 
dictionaries, this option was not appropriate for this student population because even though they 
spoke Hmong as their first language, most of them read and wrote more fluently in English. There are 
other populations with similar characteristics. The choice of an English only dictionary fits these 
populations as suggested by the survey results of the LEP group, which showed that most used 
English dictionaries in classes (90.7%). Of course, were students literate in their first language they 
would possibly benefit more from a high quality bilingual Hmong/English dictionary. However, there 
are many languages for which bilingual dictionaries are not available. Furthermore, the cost 
effectiveness of offering bilingual dictionaries for every language group is a problematic ideal not 
uncommon to similar accommodations such as translated tests or test instructions. 



A factor in this study that may have contributed to the general outcome was test time constraints. To 
work around regular schedules across sites, two methods for managing the return of students to 
classrooms were used. These could have affected the length of time taken to finish the tests or 
influenced the outcome of spending significantly less time on the second half of the test. At one 
school, the policy was that students would remain in the testing room until all students were finished. 
At the two other schools, each student was allowed to leave after he or she completed the testing 
materials. Further, other student factors such as time management skills during test-taking, peer 
pressure to finish, the test not having as much real world significance as the real Basic Standards 
Test administration, or students wanting to return to friends in other classes could have influenced the 
overall outcome. It is difficult to determine what possible effect, if any, these student release policies 
may have had on test results given. 
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It may also be a consideration that in studies of this type dictionaries and passages are inadvertently 
chosen that are more appropriate for the middle to upper intermediate level students, which is 
consistent with the nature of the tests used in standards-based assessments. If the level of reading 
text and dictionary are set at an easier ability level, the lower intermediate level students may benefit 
more, and the higher intermediate level would benefit less. This is similar to the advanced student not 
benefiting from the dictionary in this study. It is also likely that a general vocabulary threshold 
prevents lower level ability students from using a basic level dictionary with meaningful benefit. 

These challenges of choosing an appropriate dictionary and writing items that take into consideration 
the possibility that a dictionary accommodation may be made available help to highlight some of the 
issues involved in allowing dictionaries on reading tests. Some researchers and test developers may 
still argue that dictionaries give away answers, or that reading comprehension includes vocabulary 
knowledge, making the difficulty added by the vocabulary load of a passage appropriate. Others may 
argue that specific vocabulary knowledge is less critical to demonstrating reading for comprehension, 
and that strategic competence (how students use their knowledge) should also be assessed in 
reading tests (Bachman, 1996). Of these, some may argue that reading skill should include the 
abilities needed to effectively use a dictionary. Those choosing this option should be prepared to 
make decisions about different kinds of dictionaries, varying levels of student language and dictionary 
proficiency, and the specific interactions between test items, test passages, and dictionary entries. 



Recommendations 

While the simplified English dictionary accommodation may be helpful for intermediate level LEP 
students who have some dictionary skills, as shown in this study, students whose English proficiency 
falls within lower and higher ability levels would not be expected to benefit as much from the 
accommodation. This finding, perhaps more than any other, highlights the need for decisions about 
accommodations to be made on an individual basis- not for students as a group, but for students who 
need a specific accommodation. The current research also leads to several recommendations. 

• It is important for test administrators and developers to consider whether dictionaries will be 
made available, how the availability of the dictionary may interact with the test, and what 
specific language skills are to be tested when writing test items. It is unlikely that new 
dictionaries would be chosen each time a test is administered, or that tests would be developed 
to align with dictionary definitions. On the other hand, one of the dictionary skills that students 
eventually gain is determining whether the information matches their needs. Students 
experiencing exposure to different dictionaries not only choose the information they use, but 
also acquire the added skills needed to take the test. It is questionable whether these added 
skills increase the difficulty more than the dictionaries reduce barriers created by limited English 
proficiency. 

• Questions about what specific skills are included within the term “reading skill” need to be 
resolved in developing assessments with a dictionary or glossary accommodation. 

• If dictionaries are used, it is important for administrators to consider the quality and 
appropriateness of a dictionary chosen for varying student populations. It is most likely not 
feasible to provide all biliterate students with dictionaries for each language represented in an 
LEP population. English glossaries may appear to be more practical from a test development 
viewpoint and cost effectiveness. This option may still raise questions about how one defines 
reading skills and insuring test validity. 

• The process of selecting a dictionary for this study revealed that test administrators should 
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ensure that the level of vocabulary in test passages and items is compatible with dictionary entries, 
assuming test administrators are controlling for dictionary quality. The process also revealed 
that certain dictionaries could actually put students at a disadvantage unless there was careful 
consideration of the match between test and dictionary. 
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Appendix A 

Pre-test Questions from Questionnaire (bubble) Form 



O >w well do you understand spoken Hmong? 
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A. Very well B. Pretty well C. Well D. Not very well E. Not well at all 

2. How well do you speak Hmong? 

A. Very well B. Pretty well C. Well D. Not very well E. Not well at all 

3. How well do you read Hmong? 

A. Very well B. Pretty well C. Well D. Not very well E. Not well at all 

4. How well do you understand spoken English? 

A. Very well B. Pretty well C. Well D. Not very well E. Not well at all 

5. How well do you speak English? 

A. Very well B. Pretty well C. Well D. Not very well E. Not well at all 

6. How well do you read English? 

A. Very well B. Pretty well C. Well D. Not very well E. Not well at all 

7. Do you read or write a language other than Hmong or English? 

A. Yes B. No 

If yes, what languages? 

8. How many years have you lived in the United States? (choose one.) 

A. Less than 1 B. 1 -3 years C. 4-6 years 

D. 7-9 years E. More than 9 



Appendix B 

Post-test Questions from Questionnaire Form 

1. How good are you at using an English-only dictionary? 

A. Very good B. Pretty good C. Good D. Not very good E. Not good at all 
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2. How often did you use the dictionary to look up words you didn’t know during the tests? (choose one.) 

A. I didn’t use it. B. For very few words C. For some of the words 

D. For most of the words E. For all the words 

3. Did you use the dictionary for “the first champions of the little league?” 

A. Yes B. No 

4. Did you use the dictionary for “precocious preteen is youngest college graduate”? 

A. Yes B. No 

5. Did you use the dictionary for “stand by your stand”? 

A. Yes B. No 

6. Did you use the dictionary for “young historians take projects to granddaddy of museums”? 

A. Yes B. No 

7. Did you open the dictionary and look inside? 

A. Yes B. No 

8. Did you look up the meanings of new words using the dictionary? 

A. Yes B. No 

9. Did you look at other information about words using the dictionary? 

A. Yes B. No 



10. Did you check the meanings of words you knew before using the dictionary? 

A. Yes B. No 



11. Would it help you to use an English-only dictionary when you take a reading test? 
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A. Yes 



B. No 



12. Do you use a dictionary in any of your school classes? 



A. Yes 



B. No 



13. In the last year, has your teacher shown you how to use an English-only dictionary? 



A. Yes 



B. No 



14. If you do use a dictionary in any of your classes, what kind of dictionary do you use the most? (Choose one.) 



A. English-only B. Bilingual 



C. Hmong-only D. Other 



Appendix C 

Post-test Dictionary Exercise 
Dictionary Use Check 

Use your dictionary to find the answers to these questions. 

The underlined words in the sentences below have more than one meaning. Use your dictionary to find the correct meaning for each 
sentence. 

1 . My mother won't let me watch television on Saturday mornings because she says I get too immersed in the cartoons. 

Write the meaning o f immersed : 

2. The bank gives a premium to customers who open a savings account. 

Write the meaning of premium : 

3. What word comes after “stale” in the dictionary? 

4. What part of speech is the word “haphazard?” (circle one answer) 

a. noun b. verb c. adjective d. adverb e. other 
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